We study mean field systems whose free energy landscape is dominated by marginally stable states. We review and develop various techniques to describe such states, elucidating their physical meaning and the interrelation between them. In particular, we give a physical interpretation of the two-group replica symmetry breaking scheme and confirm it by establishing the relation to the cavity method and to the counting of solutions of the Thouless-Anderson-Palmer equations. We show how these methods all incorporate the presence of a soft mode in the free energy landscape and interpret the occurring order parameter functions in terms of correlations between the soft mode and the local magnetizations. The general formalism is applied to the prototypical case of the Sherrington-Kirkpatrick-model where we re-examine the physical properties of marginal states under a new perspective.